Reinforcement Learning with Soft State Aggregation
نویسندگان
چکیده
It is widely accepted that the use of more compact representations than lookup tables is crucial to scaling reinforcement learning RL algorithms to real world problems Unfortunately almost all of the theory of reinforcement learning assumes lookup table representa tions In this paper we address the pressing issue of combining function approximation and RL and present a function approx imator based on a simple extension to state aggregation a com monly used form of compact representation namely soft state aggregation a theory of convergence for RL with arbitrary but xed soft state aggregation a novel intuitive understanding of the e ect of state aggregation on online RL and a new heuristic adaptive state aggregation algorithm that nds improved compact representations by exploiting the non discrete nature of soft state aggregation Preliminary empirical results are also presented
منابع مشابه
Reinforcement Learning with Soft
It is widely accepted that the use of more compact representations than lookup tables is crucial to scaling reinforcement learning (RL) algorithms to real-world problems. Unfortunately almost all of the theory of reinforcement learning assumes lookup table representations. In this paper we address the pressing issue of combining function approximation and RL, and present 1) a function approx-im...
متن کاملMinimax-Based Reinforcement Learning with State Aggregation - Decision and Control, 1998. Proceedings of the 37th IEEE Conference on
One of the most important issues in scaling up reinforcement learning for practical problems is how to represent and store cost-to-go functions with more compact representations than lookup tables . In this paper , we address the issue of combining the simple function approximation method-state aggregation with minimaxbased reinforcement learning algorithms and present the convergence theory fo...
متن کاملNon-Markovian State Aggregation for Reinforcement Learning
3 Feature Reinforcement Learning 5 3.1 Feature Maps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 3.1.1 Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 3.1.2 State Aggregation and φ-uniformity . . . . . . . . . . . . . . . . . 6 3.2 Counterexamples to Open Problem 10 for V ∗ Aggregation . . . . . . . . . 8 3.2.1 Transient counterexample . . ....
متن کاملAdaptive Aggregation for Reinforcement Learning with Efficient Exploration: Deterministic Domains
We propose a model-based learning algorithm, the Adaptive Aggregation Algorithm (AAA), that aims to solve the online, continuous state space reinforcement learning problem in a deterministic domain. The proposed algorithm uses an adaptive state aggregation approach, going from coarse to fine grids over the state space, which enables to use finer resolution in the “important” areas of the state ...
متن کاملMulticast Routing in Wireless Sensor Networks: A Distributed Reinforcement Learning Approach
Wireless Sensor Networks (WSNs) are consist of independent distributed sensors with storing, processing, sensing and communication capabilities to monitor physical or environmental conditions. There are number of challenges in WSNs because of limitation of battery power, communications, computation and storage space. In the recent years, computational intelligence approaches such as evolutionar...
متن کامل